Compressed Sensing & Network Monitoring

Network monitoring and inference is an increasingly important component of intelligence gathering, from mapping the structure of the Internet, to discovering clandestine social networks, as well to information fusion in wireless sensor networks. Indeed, several international conferences are dedicated to the nascent field of network science. This article considers a particularly salient aspect of network science that revolves around large-scale distributed sources of data and their storage, transmission, and retrieval. The task of transmitting information from one point to another is a common and well-understood exercise. But the problem of efficiently sharing information from and among a vast number of distributed nodes remains a great challenge, primarily because we do not yet have well developed theories and tools for distributed signal processing, communications, and information theory in large-scale networked systems. The problem is illustrated by a simple example. Consider a network of n nodes, each having a piece of information or data xj, j=1,...n. These data could be files to be shared, or simply scalar values corresponding to node attributes or sensor measurements. Let us assume that each xj is a scalar quantity for the sake of this illustration. Collectively these data x=[x1,...,xn]T, arranged in a vector, are called networked data to emphasize both the distributed nature of the data and the fact that they may be shared over the underlying communications infrastructure of the network. The networked data vector may be very large; n may be a thousand, a million, or more. Thus, even the process of gathering x at a single point is daunting (requiring n communications at least). Yet this global sense of the networked data is crucial in applications ranging from network security to wireless sensing. Suppose, however, that it is possible to construct a highly compressed version of x, efficiently and in a decentralized fashion. This would offer many obvious benefits, provided that the compressed version could be processed to recover x to within a reasonable accuracy. There are several decentralized compression strategies that could be utilized. One possibility is that the correlations between data at different nodes are known a priori. Then distributed source Compressed Sensing & Network Monitoring